{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1> DBAdapter usage </h1>\n",
    "\n",
    "This tutorial illustrates the use of Fred2 to map gene names and retrieve database accessions and genetic or transcript sequences from a database source like BioMart. Fred2 can connect to a variety of DB sources both online and offline.\n",
    "Here, we will cover the use of Fred2 MartsAdapter as and example for online access and EnsemblAdapter for offline access."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<h2> Chapter 1: The basics </h2>\n",
    "<br/>\n",
    "We first start with importing the needed packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from Fred2.IO.MartsAdapter import MartsAdapter\n",
    "from Fred2.IO.EnsemblAdapter import EnsemblDB\n",
    "from Fred2.IO.ADBAdapter import EIdentifierTypes\n",
    "from Fred2.IO.ADBAdapter import EAdapterFields\n",
    "from Fred2.Core import Transcript"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For starters we will connect to the BioMart:\n",
    "\n",
    "Initializing the MartsAdapter, you can specify the URL under which the BioMart of your choice is to be reached by supporting the attribute **`biomart`**. If you do not choose a specific BioMart it will default to <a href=\"http://biomart.org\">http://biomart.org</a>. Here however, we will use: <a href=\"http://grch37.ensembl.org\">http://grch37.ensembl.org</a>. Please refer to the documentation of your BioMart to find the correct URL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mart_adapter = MartsAdapter(biomart=\"http://grch37.ensembl.org\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can start using the BioMart. For a comprehensive list of methods implemented to the adapter, you can refer to the <a href=\"http://fred2.readthedocs.org/en/latest/Fred2.IO.html#module-Fred2.IO.MartsAdapter\">documentation</a>.\n",
    "\n",
    "You can fetch all different kinds of sequences with the adapter. We will start with a transcript sequence to the glucagon gene. You have to provide an identifier that will be known by the BioMart and identifies the <i>transcript</i>, in our <a href=\"http://www.ensembl.org/\">ensembl</a> case in the form \"ENST...\".\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<type 'str'>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TRANSCRIPT: ENST00000375497\n",
       "\tVARIANTS:\n",
       "\tSEQUENCE: ATGAAAAGCATTTACTTTGTGGCTGGATTATTTGTAATGCTGGTACAAGGCAGCTGGCAACGTTCCCTTCAAGACACAGAGGAGAAATCCAGATCATTCTCAGCTTCCCAGGCAGACCCACTCAGTGATCCTGATCAGATGAACGAGGACAAGCGCCATTCACAGGGCACATTCACCAGTGACTACAGCAAGTATCTGGACTCCAGGCGTGCCCAAGATTTTGTGCAGTGGTTGATGAATACCAAGAGGAACAGGAATAACATTGCCAAACGTCACGATGAATTTGAGAGACATGCTGAAGGGACCTTTACCAGTGATGTAAGTTCTTATTTGGAAGGCCAAGCTGCCAAGGAATTCATTGCTTGGCTGGTGAAAGGCCGAGGAAGGCGAGATTTCCCAGAAGAGGTCGCCATTGTTGAAGAACTTGGCCGCAGACATGCTGATGGTTCTTTCTCTGATGAGATGAACACCATTCTTGATAATCTTGCCGCCAGGGACTTTATAAACTGGTTGATTCAGACCAAAATCACTGACAGGAAATAA (mRNA)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcript = mart_adapter.get_transcript_sequence('ENST00000375497', type=EIdentifierTypes.ENSEMBL)\n",
    "print type(transcript)\n",
    "\n",
    "fred2_transcript = Transcript(transcript, 'glucagon', 'ENST00000375497')\n",
    "fred2_transcript"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The adapter will yield a simple string. You can use this string to contruct your <a href=\"http://fred2.readthedocs.org/en/latest/Fred2.Core.html#module-Fred2.Core.Transcript\">transcript object</a>.\n",
    "\n",
    "The **`type`** attribute designates which type of identifier you are giving. Which ones your Adapter is supporting, you can read in the <a href=\"http://fred2.readthedocs.org/en/latest/Fred2.IO.html\">documentation</a>.\n",
    "\n",
    "\n",
    "The MartsAdapter is also sporting a method which will yield you more information on your transcript of interest than just the sequence. This is called called **`get_transcript_information`** and uses the same input as **`get_transcript_sequence`**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: '',\n",
       " 1: '-',\n",
       " 2: 'ATGAAAAGCATTTACTTTGTGGCTGGATTATTTGTAATGCTGGTACAAGGCAGCTGGCAACGTTCCCTTCAAGACACAGAGGAGAAATCCAGATCATTCTCAGCTTCCCAGGCAGACCCACTCAGTGATCCTGATCAGATGAACGAGGACAAGCGCCATTCACAGGGCACATTCACCAGTGACTACAGCAAGTATCTGGACTCCAGGCGTGCCCAAGATTTTGTGCAGTGGTTGATGAATACCAAGAGGAACAGGAATAACATTGCCAAACGTCACGATGAATTTGAGAGACATGCTGAAGGGACCTTTACCAGTGATGTAAGTTCTTATTTGGAAGGCCAAGCTGCCAAGGAATTCATTGCTTGGCTGGTGAAAGGCCGAGGAAGGCGAGATTTCCCAGAAGAGGTCGCCATTGTTGAAGAACTTGGCCGCAGACATGCTGATGGTTCTTTCTCTGATGAGATGAACACCATTCTTGATAATCTTGCCGCCAGGGACTTTATAAACTGGTTGATTCAGACCAAAATCACTGACAGGAAATAA'}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcript_info = mart_adapter.get_transcript_information('ENST00000375497', type=EIdentifierTypes.ENSEMBL)\n",
    "transcript_info                                      "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It will return a dictionary, which keys are defined by the Enum type **`EAdapterFields`**, coding for *GENE=0, STRAND=1, SEQ=2, TRANSID=3, PROTID=4*. That way, you can access the information in a more comprehensible way. For example, you will find out the strand direction of the transcript this way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'-'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcript_info[EAdapterFields.STRAND]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which will tell you that the transcript comes from the reverse strand."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are dealing with genomic positions or regions and want to find out, whether a particular one is in a gene's coding region, you can use the **`get_gene_by_position`** function to find that out. It will yield the genes name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'TP53'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mart_adapter.get_gene_by_position(17, 7565101, 7565101)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2> Chapter 2: Connecting to offline databases</h2>\n",
    "<br/>\n",
    "Fred2 Also supports the read from offline databases such as fasta and dat files as you can download from Ensebl, UniProt or RefSeq.\n",
    "To connect, you will have to initialize the corresponding adapter and feed it the location of your database file.\n",
    "\n",
    "As example, we will use the EnsemblAdapter. You can get the official sequence resources from ensembl <a href=\"http://www.ensembl.org/info/data/ftp/index.html\">here</a>. However for this tutorial, we will use a small test excerpt from the ensembl Protein sequence (FASTA)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ed = EnsemblDB()\n",
    "ed.read_seqs(\"data/Homo_sapiens.GRCh38.cds.test_stub.fa\")\n",
    "ed.read_seqs(\"data/Homo_sapiens.GRCh38.pep.test_stub.fa\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As EnsemblDB is implementing the ADBAdapter interface, you can basically achieve the same as in  <a href=\"DBAdapterUsage.ipynb#-Chapter-1:-The-basics-\">Chapter 1</a> just offline. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: 'ENSG00000138674',\n",
       " 1: '-',\n",
       " 2: 'ATGAAGTTAAAGGAAGTAGATCGTACAGCCATGCAGGCATGGAGCCCTGCCCAGAATCACCCCATTTACCTAGCAACAGGAACATCTGCTCAGCAATTGGATGCAACATTTAGTACGAATGCTTCCCTTGAGATATTTGAATTAGACCTCTCTGATCCATCCTTGGATATGAAATCTTGTGCCACATTCTCCTCTTCTCACAGGTACCACAAGTTGATTTGGGGGCCTTATAAAATGGATTCCAAAGGAGATGTCTCTGGAGTTCTGATTGCAGGTGGTGAAAATGGAAATATTATTCTCTATGATCCTTCTAAAATTATAGCTGGAGACAAGGAAGTTGTGATTGCCCAGAATGACAAGCATACTGGCCCAGTGAGAGCCTTGGATGTGAACATTTTCCAGACTAATCTGGTAGCTTCTGGTGCTAATGAATCTGAAATCTACATATGGGATCTAAATAATTTTGCAACCCCAATGACACCAGGAGCCAAAACACAGCCGCCAGAAGATATCAGCTGCATTGCATGGAACAGACAAGTTCAGCATATTTTAGCATCAGCCAGTCCCAGTGGCCGGGCCACTGTATGGGATCTTAGAAAAAATGAGCCAATCATCAAAGTCAGTGACCATAGTAACAGAATGCATTGTTCTGGGTTGGCATGGCATCCTGATGTTGCTACTCAGATGGTCCTTGCCTCCGAGGATGACCGGTTACCAGTGATCCAGATGTGGGATCTTCGATTTGCTTCCTCTCCACTTCGTGTCCTGGAAAACCATGCCAGGGGGATTTTGGCAATTGCTTGGAGCATGGCAGATCCTGAATTGTTACTGAGCTGTGGAAAAGATGCTAAGATTCTCTGCTCCAATCCAAACACAGGAGAGGTGTTATATGAACTTCCCACCAACACACAGTGGTGCTTCGATATTCAGTGGTGTCCCCGAAATCCTGCTGTCTTATCAGCTGCTTCGTTTGATGGGCGTATCAGTGTTTATTCTATCATGGGAGGTAGCACAGATGGTTTAAGACAGAAACAAGTTGACAAGCTTTCATCATCTTTTGGGAATCTTGATCCCTTTGGCACAGGACAGCCCCTTCCTCCGTTACAAATTCCACAGCAGACTGCTCAGCATAGTATAGTGCTGCCTCTGAAGAAGCCGCCCAAGTGGATTCGAAGGCCTGTTGGTGCTTCTTTTTCATTTGGAGGCAAACTGGTTACGTTTGAGAATGTCAGAATGCCTTCTCATCAGGGAGCTGAGCAGCAGCAGCAGCAGCACCATGTGTTCATTAGTCAGGTTGTAACAGAAAAGGAGTTCCTCAGCCGATCAGACCAACTTCAGCAGGCTGTGCAGTCACAAGGATTTATCAATTATTGCCAAAAAAAAATTGATGCTTCTCAGACTGAATTTGAGAAAAATGTGTGGTCCTTTTTGAAGGTAAACTTTGAGGATGATTCTCGTGGAAAATACCTTGAACTTCTAGGATACAGAAAAGAAGATCTAGGAAAGAAGATTGCTTTGGCCTTGAACAAAGTGGATGGAGCCAATGTGGCTCTTAAAGACTCTGACCAAGTAGCACAGAGTGATGGGGAGGAGAGCCCTGCTGCTGAAGAGCAGCTCTTGGGAGAGCACATTAAAGAGGAAAAAGAAGAATCTGAATTTCTACCCTCATCTGGAGGAACATTTAATATCTCTGTCAGTGGGGACATTGATGGTTTAATTACTCAGGCTTTGCTGACGGGCAATTTTGAGAGTGCTGTTGACCTTTGTTTACATGATAACCGCATGGCCGATGCCATTATATTGGCCATAGCAGGTGGACAAGAACTCTTGGCTCGAACCCAGAAAAAATACTTCGCAAAATCCCAAAGCAAAATTACCAGGCTCATCACTGCAGTGGTGATGAAGAACTGGAAAGAGATTGTTGAGTCTTGTGATCTTAAAAATTGGAGAGAGGCTTTAGCTGCAGTATTGACTTATGCAAAGCCGGATGAATTTTCAGCCCTTTGTGATCTTTTGGGAACCAGGCTTGAAAATGAAGGAGATAGCCTCCTGCAGACTCAAGCATGTCTCTGCTATATTTGTGCAGGGAATGTAGAGAAATTAGTTGCATGTTGGACTAAAGCTCAAGATGGAAGCCACCCTTTGTCACTTCAGGATCTGATTGAGAAAGTTGTCATCCTGCGAAAAGCTGTGCAACTCACTCAAGCCATGGACACTAGTACTGTAGGAGTTCTCTTGGCTGCGAAGATGAGTCAGTATGCCAATTTGTTGGCAGCTCAGGGCAGTATTGCTGCAGCCTTGGCTTTTCTTCCTGACAACACCAACCAGCCAAATATCATGCAGCTTCGTGACAGACTTTGTAGAGCACAAGGAGAGCCTGTAGCAGGACATGAATCACCTAAAATTCCGTACGAGAAACAGCAGCTCCCCAAGGGCAGGCCTGGACCAGTTGCTGGCCACCACCAGATGCCAAGAGTTCAAACTCAACAATATTATCCCCATGGAGAAAATCCTCCACCTCCGGGTTTCATAATGCATGGAAATGTTAATCCAAATGCTGCTGGTCAGCTTCCCACATCTCCAGGTCATATGCACACCCAGGTACCACCTTATCCACAGCCACAGCCTTATCAACCAGCCCAGCCGTATCCCTTCGGAACAGGGGGGTCAGCAATGTATCGACCTCAGCAGCCTGTTGCTCCTCCTACTTCAAACGCTTACCCTAACACCCCTTACATATCTTCTGCTTCTTCCTATACTGGGCAGTCTCAGCTGTACGCAGCACAGCACCAGGCCTCTTCACCTACCTCCAGCCCTGCTACTTCTTTCCCTCCTCCCCCTTCCTCTGGAGCATCCTTCCAGCATGGCGGACCAGGAGCTCCACCATCATCTTCAGCTTATGCACTGCCTCCTGGAACAACAGGTACACTGCCTGCTGCCAGTGAGCTGCCTGCGTCCCAAAGAACAGGTCCTCAGAATGGTTGGAATGACCCTCCAGCTTTGAACAGAGTACCCAAAAAGAAGAAGATGCCTGAAAACTTCATGCCTCCTGTTCCCATCACATCACCAATCATGAACCCGTTGGGTGACCCCCAGTCACAAATGCTGCAGCAACAGCCTTCAGCTCCAGTACCACTGTCAAGCCAGTCTTCATTCCCACAGCCACATCTTCCAGGTGGCCAGCCCTTCCATGGCGTACAGCAACCTCTTGGTCAAACAGGCATGCCACCATCTTTTTCAAAGCCCAATATTGAAGGTGCCCCAGGGGCTCCTATTGGAAATACCTTCCAGCATGTGCAGTCTTTGCCAACAAAAAAAATTACCAAGAAACCTATTCCAGATGAGCACCTCATTCTAAAGACCACATTTGAGGATCTTATTCAGCGCTGCCTTTCTTCAGCAACAGACCCTCAAACCAAGAGGAAGCTAGATGATGCCAGCAAACGTTTGGAGTTTCTGTATGATAAACTTAGGGAACAGACACTTTCACCAACAATCACCAGTGGTTTACACAACATTGCAAGGAGCATTGAAACTCGAAACTACTCAGAAGGATTGACCATGCATACCCACATAGTTAGCACCAGCAACTTCAGTGAGACCTCTGCTTTCATGCCAGTTCTCAAAGTTGTTCTCACCCAGGCCAATAAGCTGGGTGTCTAA'}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ed.get_transcript_information('ENST00000395310', type=EIdentifierTypes.ENSEMBL)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But you can also conduct quick exact searches to find sequence occurrences in the database.\n",
    "**`search`** will yield the first entry that contains the given sequence, ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'GAGAGA': 'ENST00000513858'}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ed.search(\"GAGAGA\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... whereas **`search_all`** will yield all entries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'GAGAGA': 'ENST00000513858,ENST00000509142,ENST00000395310,ENST00000508502,ENST00000510167,ENST00000505472,ENST00000348405,ENST00000512664,ENST00000505984,ENST00000508479,ENST00000355196,ENST00000443462,ENST00000448323,ENST00000311785,ENST00000264405,ENST00000500777,ENST00000507828'}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ed.search_all(\"GAGAGA\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
